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The  collection  systems  worked  more  or  less.  But,  there  were  no  analysts  ca¬ 
pable  of  interpreting  the  data,  since  the  Army  had  moved  to  maximum  au¬ 
tomation — and  the  automated  systems  were  not  programmed  for  so  unex¬ 
pected  a  contingency  as  a  deployment  to  an  African  backwater. 

— Ralph  Peters,  The  War  in  2020 


The  Analytic  Gap 

The  terrorist  attacks  of  11  September  2001  (9/11)  exposed  an  analyti¬ 
cal  deficiency  within  the  intelligence  community.1  However,  the  US 
Air  Force  has  invested  in  expanding  collection  capacity,  as  evidenced 
by  the  375  intelligence,  surveillance,  and  reconnaissance  (ISR)  plat¬ 
forms  added  since  9/11. 2  More  remarkably,  the  increase  in  ISR  plat¬ 
forms  has  come  at  a  time  when  the  total  Air  Force  inventory  has  de¬ 
creased  by  500  aircraft.3  ISR  platforms  such  as  the  RQ_-4,  MC-12,  and 
MQ_-1  directly  address  an  important  element  of  ISR— but  at  the  cost  of 
other  critical  elements,  such  as  intelligence  analysis.  Air  Force  Doc¬ 
trine  Document  2-0,  Global  Integrated  Intelligence,  Surveillance,  &  Recon¬ 
naissance  Operations,  defines  global  integrated  ISR  as  the  “cross-domain 
synchronization  and  integration  of  the  planning  and  operation  of  ISR 
assets;  sensors;  processing,  exploitation  and  dissemination  systems; 
and,  analysis  and  production  capabilities  across  the  globe  to  enable 
current  and  future  operations.”4  However,  today's  investment  in  collec¬ 
tion  capacity  does  not  address  the  mounting  limitations  of  intelligence 
analysis  initially  highlighted  by  the  9/11  Commission  in  2004. 5 
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Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 
VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 
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Defining  Automated  Analysis  and  Analytics 

Because  the  imbalance  between  collection  capacity  and  intelligence 
analysis  has  prevented  realization  of  true  global  integrated  ISR  opera¬ 
tions,  key  Air  Force  leaders  are  promoting  investment  in  automated  an¬ 
alytical  programs  as  a  means  of  removing  this  disparity.6  This  article 
considers  automated  analysis  the  utilization  of  algorithms  to  transform 
a  series  of  collected  data  into  a  usable  intelligence  product.  Analytics 
can  handle  immense  amounts  of  data  by  using  preprogrammed  models 
to  produce  decisive,  actionable  results.7  This  procedure  has  boosted 
profits  in  the  private  sector  by  influencing  consumers'  buying  decisions 
and  conducting  Wall  Street  transactions  in  microseconds,  processing 
more  information  and  acting  more  quickly  than  a  stock  analyst.8  Conse¬ 
quently,  the  intelligence  community  is  exploring  analytics  as  a  means 
of  managing  large  quantities  of  data,  identifying  patterns,  and  develop¬ 
ing  products  to  anticipate  the  activities  of  future  adversaries.9  A  report 
by  the  Center  for  Strategic  and  International  Studies  revealed  that  an 
analytics  program  drawing  on  data  from  public  databases  and  airline 
reservation  codes  could  have  identified  all  19  of  the  9/11  hijackers  for 
further  investigation  before  that  fateful  date.10  Such  examples  have 
driven  key  leaders  to  look  at  analytics  as  a  possible  solution  to  the  pres¬ 
ent  analytical  deficiency.  Nevertheless,  the  use  of  analytics  is  not  with¬ 
out  peril.  We  must  judiciously  approach  investing  in  analytics  to  im¬ 
prove  intelligence  and  must  fully  understand  its  advantages  and 
limitations.  Closing  the  gap  will  involve  matching  analytic  technology 
with  the  skill  set  of  the  intelligence  analyst,  thus  optimizing  the  con¬ 
cept  of  global  integrated  ISR  operations  promoted  by  the  Air  Force. 


Considerations  for  Analytics 

Evaluating  the  potential  of  analytics  must  start  with  a  determination 
of  what  the  technology  promises— specifically,  accurate  and  tailored 
intelligence  products  that  rapidly  utilize  large  quantities  of  collected 
data.  Incorporating  analytics  into  intelligence  assessment  offers  an  ap- 
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pealing  solution  to  our  analytical  shortfall  because  it  closely  mimics 
the  system  and  technology-intensive  solutions,  which  successfully  en¬ 
hanced  collection  capacity. 

Research  conducted  by  the  Massachusetts  Institute  of  Technology  re¬ 
garding  the  relationship  between  automation  and  humans  revealed 
that  automation  may  significantly  affect  a  user's  situational  aware¬ 
ness.11  Analytics  can  decrease  the  occurrence  of  common  human  bi¬ 
ases,  including  confirmation  bias  and  assimilation  bias.12  However,  it 
may  introduce  a  new  form— automation  bias,  which  describes  the  hu¬ 
man  tendency  to  accept  an  answer  generated  by  an  automated  system, 
even  in  the  presence  of  contradictory  information.13  The  likelihood  of 
such  bias  increases  with  the  complexity  of  the  task  performed  by  the 
automated  system,  primarily  because  the  user  has  difficulty  determin¬ 
ing  the  major  factors  and  processes  that  generate  the  automated  solu¬ 
tion.14  Without  this  understanding,  the  user  cannot  reliably  and  confi¬ 
dently  determine  the  accuracy  of  that  solution.  Further,  the  effects  of 
an  adversary's  cyber  attack  on  our  automated  systems  could  go  unno¬ 
ticed,  directly  affecting  the  accuracy  and  reliability  of  global  integrated 
ISR  operations  as  a  whole.  The  concept  of  automation  bias  raises  a  crit¬ 
ical  question:  will  more  use  of  analytics  close  the  gap  between  collec¬ 
tion  and  analysis  at  the  cost  of  increased  bias? 

A  second  significant  risk  associated  with  automated  analysis  and  an¬ 
alytics  concerns  the  lack  of  analytical  agility  associated  with  the  coded 
algorithms.  The  term  analytical  agility  denotes  the  ability  of  analysts, 
human  or  automated,  to  adapt  their  processes,  given  the  introduction 
of  new  evidence  or  a  paradigm  shift.  Analytically  agile  analysts  and 
systems  can  quickly  adapt  assumptions  and  processes  to  changing  in¬ 
formation  or  new  environments.  Conversely,  those  who  lack  such  agil¬ 
ity  dismiss  new  evidence  or  label  new  information  incorrectly  to  re¬ 
main  within  preestablished  concepts.  For  a  classic  example  of  an 
individual  without  analytical  agility,  we  need  only  look  to  the  Cold 
War  analyst  who  ignored  the  changing  paradigms  of  global  terrorism 
that  led  up  to  and  followed  the  events  of  9/11. 
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We  must  consider  the  significance  of  analytical  agility  when  we  eval¬ 
uate  the  utility  of  increased  analytics  within  global  integrated  ISR  op¬ 
erations.  Developing  the  algorithms  that  comprise  analytics  can  prove 
time  consuming.  Additionally,  humans  do  the  programming,  influ¬ 
enced  by  their  own  biases  and  assumptions  they  believed  accurate  and 
reliable  at  the  moment  of  creation.  Historically,  we  can  attribute  many 
intelligence  failures  to  a  lack  of  imagination  or  analytical  agility  neces¬ 
sary  to  identify  new  major  factors  and  prevent  surprise.15  Analytics  de¬ 
signers  should  be  careful  not  only  to  look  for  previously  observed  signa¬ 
tures  but  also  to  consider  emerging  signatures  or  a  possible  change  in 
the  importance  of  an  adversary's  existing  signatures.  Otherwise,  analyt¬ 
ics  may  hinder  the  discovery  of  evidence  or  misinterpret  the  contextual 
relevance  of  evidence  that  could  lead  to  realization  of  an  alteration  in 
enemy  tactics.  Assumptions  will  change;  therefore,  we  must  update 
coding  quickly  to  keep  pace  with  a  dynamic  opponent. 

Even  without  considering  the  fiscal  constraints  of  modern  military 
budgets,  simply  adding  more  human  analysts  can  never  match  the  ex¬ 
pansive  collection  capacity  of  the  Air  Force.  Innovative  technology  so¬ 
lutions  such  as  analytics  have  been  profitable  in  the  private  sector  and 
may  have  even  greater  potential  value  for  intelligence.  However,  we 
must  consider  and  weigh  such  risks  as  automation  bias  and  a  paucity 
of  analytical  agility  if  we  wish  to  invest  effectively  and  improve  the 
analytical  capability  resident  within  global  integrated  ISR  operations. 


Recommended  Investment  Strategy 

Effective  investment  in  these  operations  should  concentrate  on  clos¬ 
ing  the  gap  between  the  Air  Force's  current  collection  and  analytical 
capabilities.  Complete  reliance  on  analytics  and  automated  solutions 
to  improve  intelligence,  however,  represents  a  dangerous  approach  to 
solving  that  problem.  The  impact  of  automation  bias  and  insufficient 
analytical  agility  associated  with  analytics  could  sideline  human  ana¬ 
lysts,  relegating  them  to  rubber-stamping  an  automated  assessment 
that  reflects  little  understanding  of  accuracy  or  implications.  But 
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launching  additional  analysts  who  have  no  training  in  leveraging  ana¬ 
lytics  effectively  against  an  ever-expanding  reservoir  of  data  is  an  even 
bleaker  proposition.  Instead,  Air  Force  leadership  must  develop  a  bal¬ 
anced  investment  strategy  that  includes  analytics  and,  more  impor¬ 
tantly,  the  training  of  all-source  human  intelligence  analysts  in  exploit¬ 
ing  various  techniques  that  will  allow  them  to  understand,  operate, 
and  optimize  collection  sensors  and  automated  tools. 

In  spite  of  recent  efforts  to  incorporate  different  analytical  tech¬ 
niques  into  the  core  curriculum  of  the  technical  training  school  for  in¬ 
telligence  officers  and  enlisted  professionals  at  Goodfellow  AFB,  Texas, 
most  ISR  professionals  (with  the  possible  exception  of  graduates  of  the 
USAF  Weapons  School  and  the  Air  Force  Advanced  Analysis  Course) 
are  not  adequately  conversant  in  basic  analytical  methodologies.  The 
latter  include  problem  restatement,  red  teaming,  weighted  ranking, 
computation  of  conditional  probabilities,  hypothesis  testing,  and  utility 
analysis.  These  courses  provide  a  comprehensive  set  of  tools  for  con¬ 
ducting  objective,  thoughtful  analysis.  Without  this  training,  most 
intelligence  analysts  have  to  rely  on  their  intuition,  whereas  effective 
leveraging  of  an  automated  system  demands  an  understanding  of  its 
foundational  principles  and  methodologies.  Thus,  we  can  maximize 
the  capabilities  of  analytics  only  by  offering  training  in  specific  ana¬ 
lytical  techniques  coded  in  the  programming  of  analytics.  Expanding 
such  training  to  include  the  methodologies  and  techniques  of  analytics 
would  cost  much  less  than  a  new  collection  system  and,  arguably, 
would  produce  better  results. 

This  dual-pronged  strategy  will  yield  analytically  astute,  indepen¬ 
dent  intelligence  analysts  poised  to  optimize  the  potential  of  analytics 
and  realize  the  full  capability  of  global  integrated  ISR  operations.  Inte¬ 
grating  these  analysts  with  analytics  recognizes  that  we  cannot  pursue 
either  path  separately  as  the  sole  solution.  Even  though  the  speed  of 
analytics  permits  the  processing  of  more  data,  the  human  mind— even 
with  its  limitations— can  supply  the  requisite  analytical  agility  and 
imagination  (a  formidable  task  for  analytics  alone).  Thus,  productive 
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consolidation  of  these  pursuits  can  occur  only  if  one  is  trained  and 
equipped  to  manage  an  effective  arsenal  of  analytical  capabilities. 
Training  intelligence  analysts  to  better  exploit  various  structured, 
analytical  methodologies  and  investing  in  analytics  will  enable  en¬ 
hanced  understanding,  exploitation,  and  targeting  of  the  adversary; 
improve  the  management  of  collection  capacity  and  sensor  employ¬ 
ment;  more  effectively  derive  meaning  from  the  collected  data; 
strengthen  the  assessment  of  ISR  effectiveness;  and  facilitate  informed 
decision  making— the  ultimate  objective.  © 
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